The Demand for a Sound Baseline in GPU Memory Architecture Research

Modern GPUs adopt massive multithreading and multi-level cache hierarchies to hide long operation latencies, especially off-chip memory access latencies. However, poor cache indexing and cache line allocation policy as well as a small number of miss-status handling registers (MSHRs) can exacerbate the problem of cache thrashing and cache-missrelated resource congestion. Besides, modulo address mapping among memory partitions may cause severe partition camping, resulting in underutilization of DRAM bandwidth and capacity of banked L2 cache. Furthermore, prior GPU cache bypassing studies unrealistically assume there is no limit on the number of in-flight bypassed requests, which may lead to pathological experimental results in simulation. In this work, we investigate the performance impact of the aforementioned factors and demonstrate the necessity for a sound baseline in GPU memory architecture research. Our results show that advanced cache indexing functions can greatly reduce conflict misses and improve cache efficiency; the allocation-on-fill policy brings a better performance than allocation-on-miss. Besides, the performance does not consistently improve with more MSHRs. Instead, there can be performance degradation in certain scenarios. In addition, Xor mapping can greatly mitigate the problem of memory partition camping. Furthermore, the fact that a limited number of inflight bypassed requests can be supported should be taken into account in GPU cache bypassing studies, for more reliable results and conclusions.